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DISCLOSURE TEXT: 

- Described is a method of applying, monitoring and recording multiple test criteria for the 
efficient simultaneous characterization of performance envelope data for multiple units under 
test. - The multiple test criteria method provides an optimized approach for estimating multiple 
boundary conditions along multiple independent unidirectional vectors where each test is 
applied empirically and simultaneously to all units under test. Through the use of this 
optimized testing method, stress data can be obtained quickly, thereby enabling the user to 
evaluate design capabilities. - In prior art, manual and/or exhaustive testing has been used to 
determine the operating space or the performance envelope for the unit under test. This 
envelope was determined by empirically applying test criteria to the unit under test while 
various stress parameters were applied. Mapping the pass or fail response of the unit under 
test was then performed in order to obtain an estimate of the pass/fail boundary of the product. 
Typically, this procedure was performed manually on one unit at a time so as to apply one or 
more test criteria sequentially. As automation became evident, the manual one at a time 
approach became obsolete. Software was developed which exhaustively maps the response 
of each unit under test at each stress condition. However, this approach was considered 
inefficient since the brute-force algorithm led to many meaningless applications of the test and 
was time consuming. Also, this approach lacked the means to obtain separate data for 
different test criteria. For example, if separate performance envelopes were being obtained for 
a product running test criteria A and results were also required for test criteria B, the entire test 
had to be run separately for each of these test criteria. - The implementation of the multiple 
test criteria method consists of the disclosed algorithms used to control automated test 
equipment. The algorithms make use of the stress vector concept where the vector involves a 
combination of stresses. A vector may be thought of as a stress parameter where the 
controlled setting is under the control of an algorithm. The vector is incremented in 
predetermined steps by the algorithm, and all chosen criteria are applied until all units under 
test fail their criteria. - The following list outlines the features provided in the multiple test 
criteria method: - Selectable Number of Pass/Fail Retries - The user is able to select any 
number of pass or fail events which must occur in a row in order for the result to be considered 
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valid. This allows the filtering of intermittent or spurious results. - - Selectable Combination of 
Search Types - The user may select one of several search algorithms which include linear 
searches, binary searches and a combination of linear/binary and binary/linear searches. The 
combination searches use one search algorithm for the gross portion of the test and another 
for narrowing down the pass/fail boundary, thereby enabling the boundary to be located with 
greater precision. The relative effectiveness of the various search types depends on many 
factors, including the number of samples tested, the number of test criteria and the expected 
consistency of the results. Providing flexibility in the search types allows the user to select the 
most efficient search for the application. - - Return to Nominal for Detection of Multiple Failure 
Points - The algorithm allows for a return to normal situation in which the stress factors are 
optionally returned to nominal values until the case of interest is ready to run. This allows the 
determination of many boundary points beyond the initial failing point where a simple algorithm 
would not allow the test to continue. For example, it may be desirable to complete a power on 
self test (POST) prior to the execution of diagnostic test cases under the control of an 
operating system. - - Simultaneous Characterization - Because the results for all test criteria 
are determined simultaneously, the results are not affected by external factors, such as time of 
day, variances in stress equipment, etc. Therefore, results can be compared with confidence 
that any differences noted are significant. - - Expandability - The algorithms can be applied to 
any number of samples with any number of test criteria in use. This improves the efficiency as 
the complexity of the test scenario increases. - - Advanced Test Data Storage, Updating and 
Output Features - Because actual measured data is obtained in real time rather than a pass or 
fail indication for predetermined stress points, the output flexibility is increased. This includes 
the following: - Detection and Differentiation of Multiple Failure Modes - Because both reported 
and hangup failures are detected and tracked separately, a better understanding of the failure 
is attained. - - Separate Data for Last Pass and First Fail Data - As the test proceeds, the 
worst-case pass data and the best-case fail data for each test criteria will be constantly 
updated. This allows retrieval of data from interrupt or incomplete tests and for graphical 
display of tests in progress. - - Multiple Output Formats - The final output data can be made 
available in several forms. Among these are the summary file form which contains the data in 
a form easily interpreted by a human operator and the envelope file form which contains the 
data in a form which may be input directly to a spreadsheet or graphing program. - The 
multiple test criteria method is considered applicable in other areas, such as: a) iterative 
mathematical approaches in which the operation is performed repeatedly until a certain 
condition is met; b) in data searches in which a segment of data is analyzed for the presence 
of one or more specific strings or values; and c) in automated mechanical tests such as 
hardness, breaking stress or tolerance testing. Additionally, the data storage format approach 
could be applicable in any case where many variables must be updated to reflect the state of 
many objects. The algorithm is equally useful for testing products at any level of integration. 
The flowcharts indicate a unit under test reset by the term POR for power on reset. For 
different products, another type of reset might be used. The term hangup refers to the 
detection of the condition in which the unit under test has not indicated a failure or halted 
during the running of a test case, but in which the unit under test has not reported or 
responded within the expected amount of time. This is useful in the case where a product 
hangs or locks up during the test when the hang cannot be attributed to the execution of a 
diagnostic test. - The flow charts (Figs. 1-10) show the various steps required by the multiple 
test criteria method during the processing of the algorithms. Fig. 1 shows a flow chart of the 
steps required to run the high-level run-time algorithm. The remaining flow charts, Figs. 2 to 
1 0, show the steps to run the algorithms of each main sub-process. The best way to follow the 
flow charts is to visualize several of the units under test. For example, to determine the limits 
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to which the system can be stressed before failures occur requires the determination of the 
exact failure point at which each diagnostic test begins to fail on each of the units under test 
as the stress increases. The descriptions of the various flow charts are as follows: Fig. 1 - 
Runtime. Flo - This is a global process. At the completion of this process, all equipment is 
initialized. A complex series of events now occurs. In general, the method sets a stress 
condition, applies a test criterion, logs the results of the test and then determines the next 
stress condition to be set based on the results of the test. The algorithm allows simultaneous 
determination of many criteria for many systems. This process continues until all desired data 
is obtained, after which all equipment is shut down. Within the global process are several 
important sub-processes. Each of these is shown in its own flow chart (Figs. 2 to 10) and 
described below. The descriptions are simplified in that many additional checks and 
adjustments are made during the course of each process. Fig. 2 - INITVAR.FLO - In this sub- 
process, all data arrays are set up and initialized and the starting point is determined. Fig. 3 - 
INITVEC.FLO - In this sub-process, the initial stress point is programmed into the test 
equipment and verified. Then the units under test are started. Fig. 4 - PROCLOOP.FLO - In 
this looping sub-process, each unit under test is examined in sequence. One of several 
possible statuses is determined for each unit under test. If a system is determined to have 
failed a test, this information is logged. The loop continues until the status of all tests on all 
units under test have been determined. Fig. 5 - PROCPAR.FLO - In this sub-process, 
communications from the units under test are interpreted and reacted to. The units under test 
can be sending one of three types of messages to the control system. The first two of these 
are the messages that a particular test passed or failed. When either of these occurs, the data 
points associated with that test on that unit under test are updated. The other possible 
message is a request for permission to run a particular test. When this occurs, the control 
system consults several rules, taking into account whether the operator has selected that test, 
whether that test has failed previously, etc., and responds appropriately to the unit under test. 
This sub-process also tracts and reacts to the status unit under test, such as if each system 
has completed its test, then the next step is initiated. Fig. 6 - CHKDONE.FLO - This process 
determines whether a particular test vector has been completed for any units under test and 
returns its decision to the main processor. At each invocation, it determines if any unit under 
test can be eliminated from future tests. When all units under test have been eliminated, the 
vector is complete. Fig. 7 - NEXTSTEP. FLO - This sub-process resets all data arrays and 
prepares the test system to run the tests at the next stress point. Figs. 8 & 9 - 
RECOVER.FLO/RECNOM.FLO - These sub-processes occur when a unit under test has 
hung. They conditionally bring the stress conditions back to nominal and reset the unit under 
test, allowing it to recover from the hang. Fig. 10 - WAITPOST.FLO - This is a special sub- 
process which occurs between the time that the units under test are initialized and the time 
that they report that they have successfully brought themselves up. Hangups or failures during 
initialization are detected here. - Not shown in the flow charts are the following periodic 
checks, all of which halt the algorithms upon failure: - Every five seconds, read the status of all 
connected equipment, such as temperature chambers and power supplies. Halt upon report of 
abnormal status. - Every five seconds, read the current values from all connected equipment 
and compare each value read with the acceptable range of equipment. Halt if any parameter 
exceeds the acceptable equipment ranges. - Every sixty seconds, read all data logger 
channels and compare each value read with the range of the equipment unit connected to the 
channel. Halt upon detection of an out of range condition. This provides an independent check 
on the current stress values being applied to the units under test. - Every five seconds, read 
current temperatures from temperature chambers and compare each value read with the 
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desired value of the temperature chamber. This assumes that the temperatures have not gone 
out of control. 
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